Techniques for immersive virtual reality experiences

ABSTRACT

Various embodiments of the invention disclosed herein provide techniques for generating a three-dimensional virtual environment. A 3D object/effects generator executing on a client device receives a media content item. The 3D object/effects generator receives content metadata associated with the media content item. The 3D object/effects generator generates at least a portion of a 3D virtual environment based on the content metadata. The 3D object/effects generator displays the media content item and the at least a portion of the 3D virtual environment on a display associated with a virtual reality device.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates generally to virtual reality systems and,more specifically, to techniques for immersive virtual realityexperiences.

Description of the Related Art

Virtual reality (VR) is a rapidly growing form of media entertainment.In a VR system, a user experiences an artificial three-dimensional (3D)environment generated by a computer. The user views the VR environmentvia a display apparatus, such as VR glasses or a VR headset. The displayapparatus may be fitted with speakers or other auditory devices togenerate sounds. Further, the display apparatus may be fitted withhaptic feedback devices to generate tactile and kinesthetic stimulus.The virtual reality system also may include various sensors, such ascameras, microphones, accelerometers, and gyroscopes, in order toreceive oral, physical, and visual inputs from the user. In this manner,the user is able to navigate through a simulated 3D environment completewith sight, sound, and touch.

In some applications, a user can view two-dimensional (2D) mediacontent, such as movies, television programs, or other video sources,while experiencing the virtual 3D environment. In such cases, the VRsystem typically generates a fixed 3D environment, such as a livingroom, and projects the 2D media content onto a virtual display screenthat is present in the fixed 3D environment. The user then views the 2Dmedia content on the virtual display screen within the virtual 3Denvironment. One drawback to this approach is that the interactionbetween the user and the VR environment is typically quite minimal.Consequently, the user experience is oftentimes static, passive, andessentially the same as watching a movie or television program in one'sown living room. As a result, the user does not enjoy the rich,immersive 3D experience that VR systems are capable of generating andproviding.

As the foregoing illustrates, what is needed in the art are moreeffective techniques for generating immersive environments when viewing2D media content via a VR system.

SUMMARY OF THE INVENTION

Various embodiments of the present application set forth acomputer-implemented method for generating a three-dimensional virtualenvironment. The method includes receiving a media content item. Themethod further includes receiving content metadata associated with themedia content item. The method further includes generating at least aportion of a 3D virtual environment based on the content metadata. Themethod further includes displaying the media content item and the atleast a portion of the 3D virtual environment on a display associatedwith a virtual reality device.

Other embodiments of the present invention include, without limitation,a computer-readable medium including instructions for performing one ormore aspects of the disclosed techniques, as well as a computing devicefor performing one or more aspects of the disclosed techniques.

At least one advantage of the disclosed techniques is that 2D mediacontent, when viewed via a VR headset, is more interactive andcustomizable relative to prior approaches. More particularly, bypresenting appropriate 3D elements in the VR environment and providingcustomized selections of different aspects of the VR environment, theuser experience is more immersive and unique relative to priorapproaches. As a result, the user has the experience of being “inside”the 2D media content rather than just viewing the content from an“outsider's” perspective. These advantages represent one or moretechnological improvements over the prior art.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the inventioncan be understood in detail, a more particular description of theinvention, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIG. 1 illustrates a system configured to implement one or more aspectsof the present invention;

FIG. 2 is a more detailed illustration of the content management serverof FIG. 1, according to various embodiments of the present invention;

FIG. 3 is a more detailed illustration of the client device of FIG. 1,according to various embodiments of the present invention;

FIGS. 4A-4D illustrate various examples of an immersive VR environmentgenerated by the system of FIG. 1, according to various embodiments ofthe present invention;

FIGS. 5A-5C illustrate example graphical user interface (GUI) screensfor customizing the immersive VR environment generated by the system ofFIG. 1, according to various embodiments of the present invention; and

FIGS. 6A-6C set forth a flow diagram of method steps for generating animmersive VR environment, according to various embodiments of thepresent invention.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth toprovide a more thorough understanding of the present invention. However,it will be apparent to one of skill in the art that embodiments of thepresent invention may be practiced without one or more of these specificdetails.

System Overview

FIG. 1 illustrates a system 100 configured to implement one or moreaspects of the present invention. As shown, the system includes, withoutlimitation, a content management server 102, a client device 104, a VRheadset 106, and a 3D object database 108 in communication with eachother via a network 120. Network 120 may be any suitable environment toenable communications among remote or local computer systems andcomputing devices, including, without limitation, wireless and wiredLANs (Local Area Networks) and internet-based WANs (Wide Area Networks).

The content management server 102, also referred to herein as a contentmanagement system (CMS), includes, without limitation, a computingdevice that may be a standalone server, a cluster or “farm” of servers,one or more network appliances, or any other device suitable forimplementing one or more aspects of the present invention.Illustratively, content management server 102 communicates over network120 via communications link 132.

As further described herein, content management server 102 receivesvirtual environment data associated with a particular item of 2D mediacontent, such as a movie, television program, or other video content. Insome embodiments, content management server 102 may receive the virtualenvironment data from a user, such as a production assistant orpost-production assistant. The user may enter the virtual environmentdata via a graphical user interface (GUI), an electronic form, or viaany other technically feasible approach. The virtual environment dataincludes information that refers to the 2D media content as a whole,such as a description of the virtual environment and the movie buddyassociated with the movie. The virtual environment data also includesinformation regarding effects that occur at particular points in timeduring the movie, including, without limitation, color effects, 3Dobject effects, environment vibration effects, and screen size,location, and vibration effects.

Upon receiving the virtual environment data, content management server102 generates corresponding 3D content metadata. Content managementserver 102 may generate and store the 3D content metadata concurrentlywith or shortly after receiving the virtual environment data. Then, whenclient device 104 requests a movie, television program, or other videocontent, content management server 102 may retrieve and transmit the 3Dcontent metadata to client device 104. Alternatively, content managementserver 102 may store the virtual environment data in memory or storage.Then, client device 104 requests a movie, television program, or othervideo content, content management server 102 may retrieve thecorresponding virtual environment data, generate the 3D contentmetadata, and transmit the 3D content metadata to the client device 104.The 3D content metadata may be in any technically feasible format,including, without limitation, JavaScript Object Notation (JSON),eXtensible Markup Language (XML), and HyperText Markup Language (HTML).

Client device 104 includes, without limitation, a computing device thatmay be a personal computer, video game console, personal digitalassistant, mobile phone, mobile device, or any other device suitable forimplementing one or more aspects of the present invention.Illustratively, client device 104 communicates over network 120 viacommunications link 134. Communications link 134 may be any technicallyfeasible communication channel, including, without limitation,point-to-point communications channels, one or more LANs, and one ormore internet-based WANs. Further, client device 104 communicates withVR headset 106 over any technically feasible wired or wirelesscommunications protocol via communications link 136.

As further described herein, client device 104 receives user requests toview2D media content items, such as movies, television programs, orother video content. Upon receiving a user request to view a 2D mediacontent item, client device 104 transmits the request to content mediaserver 102. In response, content management server 102 transmits the 2Dmedia content to client device 104. In addition, content managementserver 102 transmits the corresponding 3D content metadata to clientdevice 104. Based on the 3D content metadata, client device retrievesdescriptions of any needed 3D content items from 3D object database 108.From the 3D content metadata and 3D objects, client device 104 generatesa 3D environment and any other related 3D objects to create a 3D virtualenvironment. Client device 104 further generates a virtual screen withinthe 3D virtual environment and displays the 2D media content on thevirtual screen. Client device 104 further generates a virtual moviebuddy who watches the 2D media content with the user. Client device 104translates the generated 2D and 3D visual elements into the nativeformat of VR headset 106. Client device 104 transmits the translated 2Dand 3D visual elements for display.

VR headset 106 includes, without limitation, a computing device that maybe a personal computer, video game console, personal digital assistant,mobile phone, mobile device, or any other device suitable forimplementing one or more aspects of the present invention.Illustratively, VR headset 106 communicates with client device 104 overany technically feasible wired or wireless communications protocol viacommunications link 136. VR headset 106 receives translated 2D and 3Dvisual elements from client device 104. VR headset 106 displays thereceived 2D and 3D visual elements on a display screen associated withVR headset 106.

In addition or alternatively, VR headset 106 communicates over network120 via communications link 138. Communications link 138 may be anytechnically feasible communication channel, including, withoutlimitation, point-to-point communications channels, one or more LANs,and one or more internet-based WANs. Via communications link 138, VRheadset 106 performs any one or more functions described in conjunctionwith client device 104, as further described herein.

3D object database 108 includes, without limitation, a storage devicethat may be a standalone networked attach storage (NAS) system, astorage area-network (SAN), a cluster or “farm” of storage devices, adistributed storage architecture, or any other device suitable forimplementing one or more aspects of the present invention. In additionor alternatively, 3D object database 108 may include, withoutlimitation, a computing device with a storage subsystem that may be astandalone server, a cluster or “farm” of servers, one or more networkappliances, or any other device suitable for implementing one or moreaspects of the present invention. Illustratively, 3D object database 108communicates over network 120 via communications link 140. 3D objectdatabase 108 stores 3D virtual environment data, 3D object data, moviebuddy data, and any other information to perform one of more of thetechniques disclosed herein.

It will be appreciated that the system shown herein is illustrative andthat variations and modifications are possible. In one example, althoughthe system 100 of FIG. 1 is illustrated with one content managementserver 102, one client device 104, one VR headset 106, and one 3D objectdatabase 108, the system 100 could include any technically feasiblenumber of content management servers 102, client devices 104, VRheadsets 106, and 3D object database 108 within the scope of the presentdisclosure. In another example, the 2D content items, virtualenvironment data and 3D content metadata are described as being storedon content management server 102. Similarly, 3D virtual environmentdata, 3D object data, movie buddy data are described as being stored on3D object database 108. However, any data could be stored on anytechnically feasible computing device or storage device that is incommunication with network 120.

In another example, the techniques described herein could be used inconjunction with live events, such as sporting events. In such cases, aproduction assistant or an automated statistics data system couldprovide data and markers on the occurrence of specific key eventsoccurring during the live event. Content management server 102 generates3D content metadata corresponding to the data and markers, and transmitsthe 3D content metadata to client device 104. In this manner, contentmanagement server 102 could generate and transmit the 3D contentmetadata as a live data feed in conjunction with the live video feed.The 3D content metadata could identify key events, such as a first down,touchdown, or field goal in American football, to trigger environmental,screen, and buddy effects. The movie buddy could be the user's favoritefootball player. Alternatively, the movie buddy could be a default buddyselected from one of the competing teams or from the set of announcersfor the game. The environment could be a simulated arena or stadiumfilled with cheering fans.

In addition, one of the user selections could be the team that the useris rooting for. When the selected team scores, content management server102 transmits corresponding 3D content metadata to client device 104.Client device generates appropriate effects in the 3D virtualenvironment. Such effects could include, without limitation, the crowdin the environment that rises and cheers, fireworks appears overhead,your movie buddy jumps up and down with the user, and the screenvibrates and expands. These effects could be programmed prior to thelive event. The effects are then triggered when the client devicereceives the live 3D content metadata.

In yet another example, the 3D virtual environment, movie buddy, andvarious effects could be supplied by a third party in addition to oralternatively to such content being supplied by the original producer ofthe 2D media content. In this manner, the director or an actor in amovie could appear as the movie buddy and provide commentary as themovie is being displayed on the virtual screen. A third party moviebuddy could appear and provide humorous or informative information andadditional monologue or dialogue as the movie is being displayed. Suchthird party content provides additional options for the 3D virtualenvironment that could be available for selection by the user whenviewing 2D content. The user could select a movie and separately select3D content metadata and 3D objects that are independently created bythird parties.

Several techniques for generating and displaying 3D virtual objects inconjunction with 2D media content are now described in further detail.

VR System for Generating Immersive Virtual Reality Experiences

FIG. 2 is a more detailed illustration of the content management server102 of FIG. 1, according to various embodiments of the presentinvention. As shown, content management server 102 includes, withoutlimitation, a central processing unit (CPU) 202, storage 204, aninput/output (I/O) devices interface 206, a network interface 208, aninterconnect 210, and a system memory 212.

The processor 202 retrieves and executes programming instructions storedin the system memory 212. Similarly, the processor 202 stores andretrieves application data residing in the system memory 212. Theinterconnect 210 facilitates transmission, such as of programminginstructions and application data, between the processor 202,input/output (I/O) devices interface 206, storage 204, network interface208, and system memory 212. The I/O devices interface 206 is configuredto receive input data from user I/O devices 222. Examples of user I/Odevices 222 may include one of more buttons, a keyboard, and a mouse orother pointing device. The I/O devices interface 206 may also include anaudio output unit configured to generate an electrical audio outputsignal, and user I/O devices 222 may further include a speakerconfigured to generate an acoustic output in response to the electricalaudio output signal. Another example of a user I/O device 222 is adisplay device that generally represents any technically feasible meansfor generating an image for display. For example, the display devicecould be a liquid crystal display (LCD) display, CRT display, or DLPdisplay. The display device may be a TV that includes a broadcast orcable tuner for receiving digital or analog television signals.

Processor 202 is included to be representative of a single CPU, multipleCPUs, a single CPU having multiple processing cores, and the like. Andthe system memory 212 is generally included to be representative of arandom access memory. The storage 204 may be a disk drive storagedevice. Although shown as a single unit, the storage 204 may be acombination of fixed and/or removable storage devices, such as fixeddisc drives, floppy disc drives, tape drives, removable memory cards, oroptical storage, network attached storage (NAS), or a storagearea-network (SAN). Processor 202 communicates to other computingdevices and systems via network interface 208, where network interface208 is configured to transmit and receive data via a communicationsnetwork.

The system memory 212 includes, without limitation, a content metadatacollection engine 232, 2D media content 242, and a content metadatadatabase 244. The content metadata collection engine 232, when executedby the processor 202, performs one or more operations associated withthe content management server 102 of FIG. 1, as further describedherein. In operation, content metadata collection engine 232 receivesvirtual environment data and generates corresponding 3D contentmetadata.

The virtual environment data identifies the corresponding 2D mediacontent. The virtual environment data further includes various detailsregarding the 3D virtual environment, virtual screen, and movie buddy.For example, the virtual environment data could specify sound data, suchas words spoken by the movie buddy and sound effects. Further, virtualenvironment data could specify certain times when the volume is adjustedup or down. The virtual environment data could specify color andlocation of the environment. For example, a blinking red light in the 2Dmedia content may trigger a blinking red light in the environment aswell as other chromatic effects. The virtual environment data couldspecify vibration effects for the environment and the virtual screen.For example, an explosion in the 2D media content movie could triggervibration of the virtual environment and the virtual screen.

The virtual environment data could specify the screen size and locationof the virtual screen. For example, during an intense scene in the 2Dmedia content, the virtual screen could move closer and then move backwhen the scene is complete. The virtual environment data could specify3D objects to display during playback of the 2D media content, such asthe 3D virtual environment and the movie buddy. The virtual environmentdata could further specify certain 3D objects to generate and display atspecific times during playback of the 2D media content, such as flyingdebris during an explosion or flying bugs during an appropriate scene.

The virtual environment data could also specify 3D objects to generateand display locally as well as 3D object models to download from 3Dobject database 108. The locally generated and animated 3D objects maybe simple objects, such as flying cubes, pyramids, raindrops, andbubbles, as well as complex objects, such as movie buddies and animalstravelling through the 3D virtual environment.

Content metadata collection engine 232 generates and stores the 3Dcontent metadata concurrently with or shortly after receiving thevirtual environment data. Then, when client device 104 requests a movie,television program, or other video content, metadata collection engine232 retrieves and transmits the 3D content metadata to client device104. Alternatively, content metadata collection engine 232 stores thevirtual environment data in memory or storage. Then, when client device104 requests a movie, television program, or other video content,content metadata collection engine 232 retrieves the correspondingvirtual environment data, generates the 3D content metadata, andtransmits the 3D content metadata to client device 104.

The generated 3D content metadata includes attributes for variousaspects of the 3D virtual environment and movie buddy associated withthe 2D media content. In one example, the 3D content metadata couldinclude a first section of 3D content describing elements thatcorrespond to the 2D media content as a whole rather than to a specifictime. Further, the 3D content metadata could include a second section ofoptical content that describes particular effects that occur at aparticular time during the display of the 2D media content. The 3Dcontent section includes a description of an ocean room-themedenvironment and a movie buddy. The optical content section includesvarious color, location, size, vibration and animated object effectsthat occur at key points as the 2D media content plays on the virtualscreen. A listing of the 3D content metadata defining these elements isillustrated in Table 1 below. Although the 3D content metadata of Table1 is presented in JavaScript Object Notation (JSON), any technicallyfeasible format may be used within the scope of the present disclosure.

TABLE 1 3D Content Metadata Example 0010 { 0020  “3d_content”: { 0030  “environment”: { 0040    “name”: “oceanRoom”, 0050    “type”:“roomSetting”, 0060    “value”:“http://cdn1.edgedatg.com/aws/3d123123.3dx”, 0070    “quality”: “high”,0080    “format”: “3dx”, 0090    “id”: “99” 0100   }, 0110  “movie_buddy”: { 0120    “name”: “mybuddy”, 0130    “type”:“defaultBuddy”, 0140    “animated”: true, 0150    “sound”: true, 0160   “value”: “http://cdn1.edgedatg.com/aws/3dbwww123.3dx”, 0170   “quality”: “high”, 0180    “format”: “3dx”, 0190    “id”: “100” 0200  } 0210  }, 0220  “optical_content”: { 0230   “color”: { 0240   “type”: “redFlashingLights”, 0250    “r_value”: 255, 0260   “g_value”: 0, 0270    “b_value”: 0, 0280    “duration”: 3000, 0290   “startTime”: 234343, 0300    “fadeout”: true, 0310    “fadeoutStart”:2500, 0320    “fadein”: true, 0330    “fadeinStop”: 500, 0340    “id”:“101” 0350   }, 0360   “environment_vibration”: { 0370    “type”:“upAndDown”, 0380    “duration”: 2300, 0390    “startTime”: 678343, 0400   “id”: “103” 0410   }, 0420   “3d_effects”: { 0430    “3d_effect”: [0440     { 0450      “name”: “flyingCube”, 0460      “type”:“explosionEffect”, 0470      “generateNew”: false, 0480      “value”:“http://cdn1.edgedatg.com/aws/cube1.3dx”, 0490      “quality”: “low”,0500      “format”: “3dx”, 0510      “type”: “upAndDown”, 0520     “duration”: 2300, 0530      “startTime”: 678343, 0540     “start_position”: “14, 343, 12”, 0550      “end_position”: “250,343, 12” 0560     }, 0570     { 0580      “name”: “flyingCube”, 0590     “type”: “explosionEffect”, 0600      “generateNew”: false, 0610     “value”: “http://cdn1.edgedatg.com/aws/cube2.3dx”, 0620     “quality”: “low”, 0630      “format”: “3dx”, 0640      “type”:“upAndDown”, 0650      “duration”: 2300, 0660      “startTime”: 678343,0670      “start_position”: “12, 43, 12”, 0680      “end_position”:“150, 343, 12” 0690     }, 0700     { 0710      “name”: “flyingCube”,0720      “type”: “explosionEffect”, 0730      “generateNew”: false,0740      “value”: “http://cdn1.edgedatg.com/aws/cube3.3dx”, 0750     “quality”: “low”, 0760      “format”: “3dx”, 0770      “type”:“upAndDown”, 0780      “duration”: 2300, 0790      “startTime”: 678343,0800      “start_position”: “22, 343, 12”, 0810      “end_position”:“250, 343, 12” 0820     }, 0830    ], 0840    “count”: 3, 0850    “id”:“2052377” 0860   }, 0870   “screen_vibration”: { 0880    “type”:“sideToSide”, 0890    “duration”: 2300, 0900    “startTime”: 678343,0910    “id”: “104” 0920   }, 0930   “screen_location”: { 0940   “type”: “closeUp”, 0950    “duration”: 2300, 0960    “startTime”:98343, 0970    “startingTransition”: 300, 0980    “endingTransition”:300, 0990    “id”: “105” 1000   }, 1010   “screen_size”: { 1020   “type”: “zoomIn”, 1030    “duration”: 300, 1040    “startTime”:58343, 1050    “startingTransition”: 300, 1060    “endingTransition”:300, 1070    “id”: “106” 1080   } 1090  } 1100 }

The effects described in the 3D content metadata presented in Table 1are now described in further detail. Lines 0010 and 1100 delimit the 3Dcontent metadata as specified in the intervening lines. Lines 0020-0210delimit 3D content attributes including environment attributes and moviebuddy attributes.

Lines 0030-0100 delimit the environment attributes associated with anocean-themed environment. The environment is named oceanRoom (line 0040)with an identifier of 99 (line 0090). The environment is identified as aroom setting (line 0050). The environment is specified as high quality(line 0070) and is in the 3dx file format (line 0080). The 3dx filedefining the environment is located at the URL specified in line 0060.

Lines 0110-0200 delimit the movie buddy attributes associated with amovie buddy. The movie buddy is named mybuddy (line 0120) with anidentifier of 100 (line 0190). The movie buddy is identified as thedefault buddy (line 0130). The movie buddy is animated (line 0140) andgenerates sound (line 0150). The movie buddy is specified as highquality (line 0170) and is in the 3dx file format (line 0180). The 3dxfile defining the movie buddy is located at the URL specified in line0160.

Lines 0220-1090 delimit optical content attributes including colorattributes, environment vibration attributes, 3D effects attributes,screen vibration attributes, screen location attributes, and screen sizeattributes.

Lines 0230-0350 delimit the color attributes for a flashing lighteffect. The color effect is identified as red flashing lights (line0240) with an identifier of 101 (line 0340). The red value (line 0250),green value (line 0260), and blue value (line 0270) of the color isspecified 255, 0, and 0, respectively. The red flashing lights coloreffect begins at a time of 234,343 milliseconds in the 2D media content(line 0290) for a duration of 3000 milliseconds (line 0280). The redflashing lights color effect includes a fade in (line 0320) starting atthe beginning of the effect until millisecond 500 (line 0330). The redflashing lights color effect also includes a fade out (line 0300)starting at millisecond 2500 (line 0310) until the end of the effect.

Lines 0360-0410 delimit the attributes associated with an environmentvibration effect. The environment vibration effect is identified as upand down (line 0370) with an identifier of 103 (line 0400). The up anddown environment vibration effect begins at a time of 678,343milliseconds in the 2D media content (line 0390) for a duration of 2300milliseconds (line 0380).

Lines 0420-0860 delimit the 3D effects attributes. The 3D effectsattribute section includes a single 3D effect (lines 0430 and 0830) withan identifier of 2052377 (line 0850). The 3D effect includes threecomponents (line 0840).

The first component of the 3D effect (lines 0440 and 0560) is identifiedas a flying cube (line 0450). The flying cube effect is an explosioneffect (line 0460) with up and down motion (line 0510). The flying cubemoves from a starting XYZ position of 14, 343, 12 (line 0540) to anending position of 250, 343, 12 (line 0550), indicating that the cubetravels in a horizontal (X) direction. The flying cube does not generatenew flying cubes during the effect (line 0470). The flying cube effectbegins at a time of 678,343 milliseconds in the 2D media content (line0530) for a duration of 2300 milliseconds (line 0520). The flying cubeis specified as low quality (line 0490) and is in the 3dx file format(line 0500). The 3dx file defining the flying cube is located at the URLspecified in line 0480.

The second component of the 3D effect (lines 0570 and 0690) is alsoidentified as a flying cube (line 0580). The flying cube effect is anexplosion effect (line 0590) with up and down motion (line 0640). Theflying cube moves from a starting XYZ position of 12, 43, 12 (line 0670)to an ending position of 150, 343, 12 (line 0680), indicating that thecube travels in both a horizontal (X) and a vertical (Y) direction. Theflying cube does not generate new flying cubes during the effect (line0600). The flying cube effect begins at a time of 678,343 millisecondsin the 2D media content (line 0660) for a duration of 2300 milliseconds(line 0650). The flying cube is specified as low quality (line 0620) andis in the 3dx file format (line 0630). The 3dx file defining the flyingcube is located at the URL specified in line 0610.

The third component of the 3D effect (lines 0700 and 0820) is alsoidentified as a flying cube (line 0710). The flying cube effect is anexplosion effect (line 0720) with up and down motion (line 0770). Theflying cube moves from a starting XYZ position of 22, 343, 12 (line0800) to an ending position of 250, 343, 12 (line 0810), indicating thatthe cube travels in a horizontal (X) direction. The flying cube does notgenerate new flying cubes during the effect (line 0730). The flying cubeeffect begins at a time of 678,343 milliseconds in the 2D media content(line 0790) for a duration of 2300 milliseconds (line 0780). The flyingcube is specified as low quality (line 0750) and is in the 3dx fileformat (line 0760). The 3dx file defining the flying cube is located atthe URL specified in line 0740.

Lines 0870-0920 delimit the vibration attributes associated with ascreen vibration effect. The screen vibration effect is identified asside to side (line 0880) with an identifier of 104 (line 0910). The sideto side screen vibration effect begins at a time of 678,343 millisecondsin the 2D media content (line 0390) for a duration of 2300 milliseconds(line 0800).

Lines 0930-1000 delimit the location attributes associated with a screenlocation effect. The screen location effect is identified as close up(line 0940) with an identifier of 105 (line 0990). The close up screenlocation effect begins at a time of 98,343 milliseconds in the 2D mediacontent (line 0960) for a duration of 2300 milliseconds (line 0950). Theclose up screen location effect also includes a starting transition of300 milliseconds (line 0970) and an ending transition of 300milliseconds (line 0980).

Lines 1010-1080 delimit the size attributes associated with a screensize effect. The screen size effect is identified as zoom in (line 1020)with an identifier of 106 (line 0990). The zoom in screen size effectbegins at a time of 58,343 milliseconds in the 2D media content (line1040) for a duration of 300 milliseconds (line 1030). The zoom in screensize effect also includes a starting transition of 300 milliseconds(line 1050) and an ending transition of 300 milliseconds (line 1060).

In some embodiments, the 2D media content may be prepopulated withcertain metadata in advance. In such embodiments, manual or automatedprocesses may be employed for identifying key features of the 2D mediacontent such as actors, soundtracks, locations, and purchasable items.Metadata corresponding to these key features may be generated as aseparate metadata file or embedded in the 2D media content. Contentmetadata collection engine 232 may generate 3D content metadata based onthis prepopulated metadata.

FIG. 3 is a more detailed illustration of the client device 104 of FIG.1, according to various embodiments of the present invention. As shownin client device 104 includes, without limitation, a central processingunit (CPU) 302, storage 304, an input/output (I/O) devices interface306, a network interface 308, an interconnect 310, and a system memory312. Processor 302, storage 304, input/output (I/O) devices interface306, network interface 308, interconnect 310, and system memory 312operate substantially the same as the corresponding elements describedin conjunction with FIG. 2, except as further described below.

The processor 302 retrieves and executes programming instructions storedin the system memory 312. Similarly, the processor 302 stores andretrieves application data residing in the system memory 312. Theinterconnect 310 facilitates transmission, such as of programminginstructions and application data, between the processor 302,input/output (I/O) devices interface 306, storage 304, network interface308, and system memory 312. The I/O devices interface 306 is configuredto receive input data from user I/O devices 322. Examples of user I/Odevices 322 may include one of more buttons, a keyboard, and a mouse orother pointing device. The I/O devices interface 306 may also include anaudio output unit configured to generate an electrical audio outputsignal, and user I/O devices 322 may further include a speakerconfigured to generate an acoustic output in response to the electricalaudio output signal. Another example of a user I/O device 322 is adisplay device that generally represents any technically feasible meansfor generating an image for display. For example, the display devicecould be a liquid crystal display (LCD) display, CRT display, or DLPdisplay. The display device may be a TV that includes a broadcast orcable tuner for receiving digital or analog television signals.

Processor 302 is included to be representative of a single CPU, multipleCPUs, a single CPU having multiple processing cores, and the like. Andthe system memory 312 is generally included to be representative of arandom access memory. The storage 304 may be a disk drive storagedevice. Although shown as a single unit, the storage 304 may be acombination of fixed and/or removable storage devices, such as fixeddisc drives, floppy disc drives, tape drives, removable memory cards, oroptical storage, network attached storage (NAS), or a storagearea-network (SAN). Processor 302 communicates to other computingdevices and systems via network interface 308, where network interface308 is configured to transmit and receive data via a communicationsnetwork.

The system memory 312 includes, without limitation, a 3D object/effectsgenerator 332, a native VR engine 334, a system development kit (SDK)342, and 3D object store 344. The 3D object/effects generator 332 andnative VR engine 334, when executed by the processor 302, performs oneor more operations associated with client device 104 of FIG. 1, asfurther described herein.

In operation, 3D object/effects generator 332 receives 2D media contentalong with associated 3D content metadata and 3D objects. Based on thisreceived information, 3D object/effects generator 332 generates a 3Dvirtual environment for the user to experience and navigate whileviewing the 2D media content. 3D object/effects generator 332 furthergenerates a virtual movie buddy to accompany the user and to interactwith the user. The 3D virtual environment includes a virtual screen uponwhich the 2D media content is displayed. The 3D virtual environment maybe dynamic and may be changed based on one or more of user input and the3D content metadata. For example, when the user is viewing a movie withan undersea theme, 3D object/effects generator 332 could generate anunderwater environment as the 3D virtual environment. Similarly, whenthe user is viewing a story about a boy and his toys, 3D object/effectsgenerator 332 could generate the boy's bedroom as the 3D virtualenvironment.

Similarly, the movie buddy may be dynamic and may be changed based onone or more of user input and the 3D content metadata. For example, theuser could choose a favorite character, such as the user's favoritecartoon character, as a default movie buddy. In addition oralternatively, the 3D content metadata may identify a movie buddy thatis specific to the 2D media content, such as a character from thecurrently selected 2D media content. For example, when the user isviewing a movie about a boy and his toys, 3D object/effects generator332 could generate one of the toys as the movie buddy. Similarly, whenthe user is viewing a movie about a group of superheroes, 3Dobject/effects generator 332 could generate one of the superheroes asthe movie buddy. If neither the user nor the 3D content metadata havespecified a movie buddy, then 3D object/effects generator 332 selects ageneric or default movie buddy.

In some embodiments, 3D object/effects generator 332 may detect when theuser interacts with the movie buddy. 3D object/effects generator 332detects sound uttered by thee user via a microphone associated with VRheadset 106. In addition or alternatively, 3D object/effects generator332 detects movement of the user via accelerometers, gyroscopes, orother motion sensors associated with VR headset 106. In addition oralternatively, 3D object/effects generator 332 detects input from theuser via any other technically feasible approach. In response, 3Dobject/effects generator 332 may animate the movie buddy gesture to theuser or say something to the user about the 2D media content.

As further described herein, the user may access various GUI controls toturn the movie buddy on or off, turn the environment on or off, select adifferent movie buddy or environment, select whether the movie buddyanimates or speaks, or select an option where the movie buddy andenvironment are selected based on the 2D media content being viewed.

3D object/effects generator 332 further generates various 3D effects atset time markers as specified by an appropriate time code. The time codecould be in any technically feasible format. For example, the virtualscreen could change in size and location. During a particularly intensescene, the virtual screen could move closer to eh user and then moveback when the scene completes. During a scene that includes anexplosion, the 3D virtual environment and virtual screen could vibrate.

3D object/effects generator 332 further generates 3D objects that movethroughout the 3D virtual environment at set time markers as specifiedby an appropriate time code. The time code could be in any technicallyfeasible format. 3D object/effects generator 332 stores the generated 3Dobjects in 3D object store 344. For example, during a scene in the 2Dmedia content where an explosion occurs, 3D object/effects generator 332could generate virtual flying debris that travels through the 3D virtualenvironment. Similarly, during a scene in the 2D media content where aflood occurs, 3D object/effects generator 332 could generate a floodingeffect that fills the 3D virtual environment with water. During a scenein the 2D media content that includes flying insects, 3D object/effectsgenerator 332 could generate virtual flying insects that fly around inthe 3D virtual environment.

Via the techniques described herein, 3D object/effects generator 332generates a dynamic 3D VR environment by generating, changing, andanimating objects within the environment based on the 2D media content.In some embodiments, 3D object/effects generator 332 may generateobjects locally or retrieve models of the object from 3D object database108 based on the condition of an object location flag in the 3D contentmetadata. If the object location flag indicates that the object isavailable locally, then 3D object/effects generator 332 generates theobject locally and directly. If, however, the object location flagindicates that the object is available remotely, then 3D object/effectsgenerator 332 retrieves the objects from 3D object database 108.

Upon generating the elements for display, including the virtualenvironment, virtual screen, movie buddy, and other 3D objects andeffects, 3D object/effects generator 332 accesses SDK 342. Via SDK 342,3D object/effects generator 332 translates the elements into a formatthat is native to VR headset 106. Because different VR devices andplatforms have different VR engines, SDK 342 is specific to a particularVR device or platform. Different VR devices and platforms, therefore,have different SDKs 342. Correspondingly, a particular SDK 342integrates with the native VR programming interface for thecorresponding VR device or platform. SDK 342 requests previouslygenerated 3D objects from one or both of 3D object store 344 and 3Dobject database 108. SDK 342 generates the 3D virtual environment anddisplays the 2D media content on the virtual screen within theenvironment. In some embodiments, SDK 342 may generate and animatesimple 3D objects that are specified in the 3D content metadata.

After translating the elements into the native format of VR headset 106,3D object/effects generator 332 transmits the translated elements tonative VR engine 334. In operation, native VR engine 334 transmits thetranslated elements to VR headset 106. VR headset 106 then displays thetranslated elements. In this manner, a VR headset 106 with a differentnative format may perform the techniques described herein by providing adifferent SDK 342 that supports the other native format. As a result,the capabilities of 3D object/effects generator 332 and VR headset 106capabilities may be enhanced with new features over time, and clientdevice is capable of supporting the new features.

FIGS. 4A-4D illustrate various examples of an immersive VR environmentgenerated by the system of FIG. 1, according to various embodiments ofthe present invention. As shown in FIG. 4A, an immersive VR environment400 includes a generic 3D virtual environment 406. A virtual screen 404displays 2D media content featuring two cartoon characters. A user 402experiences the 3D virtual environment 406 and views the virtual screen404 with a movie buddy 408 in the form of a cartoon character.

As shown in FIG. 4B, an immersive VR environment 420 includes a generic3D virtual environment 426. A virtual screen 424 displays 2D mediacontent featuring a scene that includes an explosion. A user 428experiences the 3D virtual environment 426 and views the virtual screen424 with a movie buddy 422 in the form of a supervillain. As shown inFIG. 4C, an immersive VR environment 440 includes a generic 3D virtualenvironment 446. 3D object/effects generator 332 generates 3D particlesand animates the 3D particles to fly from the virtual screen towards theuser.

Consequently, the 3D virtual environment 446 includes flying debris 450and 452 in the form 3D objects to the left and right of the virtualscreen 444, respectively. A user 448 experiences the 3D virtualenvironment 446 and views the virtual screen 444 with a movie buddy 442in the form of the supervillain.

As shown in FIG. 4D, an immersive VR environment 460 includes a 3Dvirtual environment 466 with an underwater theme. A virtual screen 464displays 2D media content featuring an underwater scene. 3D visualelements 470 and 472 consistent with the underwater theme travel throughthe 3D virtual environment 466 to the left and right of the virtualscreen 464, respectively. A user 468 experiences the 3D virtualenvironment 466 and views the virtual screen 464 with a movie buddy 462in the form of an aquatic cartoon character.

FIGS. 5A-5C illustrate example graphical user interface (GUI) screensfor customizing the immersive VR environment generated by the system ofFIG. 1, according to various embodiments of the present invention.

As shown in FIG. 5A, the GUI screens include a settings menu 500, amovie buddy settings menu 520, and an environment settings 540 menu. Thesettings menu 500 includes a movie buddy section 502, an environmentsection 504 and a screen animation section 506. The movie buddy section502 includes a control for turning the movie buddy on or off to enableor disable the movie buddy, respectively. The movie buddy section 502also includes a control for accessing the movie buddy settings menu 520.The environment section 504 includes a control for turning theenvironment buddy on or off to enable or disable the environment,respectively. The environment section 504 also includes a control foraccessing the environment settings menu 540.

The movie buddy settings menu 520 is accessed via the movie buddysection 502 of the settings menu 500. The movie buddy settings menu 520indicates that the currently selected movie buddy 522 is a supervillain.When selected, the change button 524 accesses the select a movie buddymenu 560, as further described herein. An animated control 526 enablesor disables whether the movie buddy is animated. A talking control 526determines whether the movie buddy is enabled or disabled to talk, suchas making comments, asking questions, and so on. A movie selectedcontrol 530 enables or disables selection of the movie buddy based onthe specific 2D media content currently being displayed.

The environment settings menu 540 is accessed via the environmentsection 504 of the settings menu 500. The environment settings menu 520indicates that the currently selected environment 542 is a sciencefiction theme. When selected, the change button 544 accesses the selectan environment menu 580, as further described herein. A room animationcontrol 526 enables or disables whether the environment is animated withcertain effects, such as vibrating, changing color, or moving. A coloreffects control 548 enables or disables certain color effects within theenvironment, such as flashing, dimming, moving, and so on. A 3D effectscontrol 548 enables or disables certain 3D object effects within theenvironment, such as flying debris, moving objects, and so on.

As shown in FIG. 5B, the select a movie buddy menu 560 is accessed viathe change button 524 of the movie buddy settings menu 520. The select amovie buddy menu 560 indicates that the currently selected movie buddy562 is a supervillain. When selected, the rate button 564 accesses arating menu (not shown) where the user can enter a rating for thecurrently selected movie buddy 562. The available movie buddy section566 illustrates the movie buddies that are available for selection. Theratings section 568 illustrates the rating for each of the availablemovie buddies.

As shown in FIG. 5C, the select an environment menu 580 is accessed viathe change button 544 of the environment settings menu 540. The selectan environment menu 580 indicates that the currently selectedenvironment 582 is a science fiction theme. When selected, the ratebutton 584 accesses a rating menu (not shown) where the user can enter arating for the currently selected environment 582. The availableenvironment section 586 illustrates the environments that are availablefor selection. The ratings section 588 illustrates the rating for eachof the available environments.

FIGS. 6A-6C set forth a flow diagram of method steps for generating animmersive VR environment, according to other various embodiments of thepresent invention. Although the method steps are described inconjunction with the systems of FIGS. 1-5C, persons of ordinary skill inthe art will understand that any system configured to perform the methodsteps, in any order, is within the scope of the present invention.

As shown, a method 600 begins at step 602, where content metadatacollection engine 232 executing on content management server 102receives and stores virtual environment data associated with aparticular item of 2D media content, such as a movie, televisionprogram, or other video content. In some embodiments, content managementserver 102 may receive the virtual environment data from a user, such asa production assistant or post-production assistant. The user may enterthe virtual environment data via a graphical user interface (GUI), anelectronic form, or via any other technically feasible approach. Thevirtual environment data includes information that refers to the 2Dmedia content as a whole, such as a description of the virtualenvironment and the movie buddy associated with the movie. The virtualenvironment data also includes information regarding effects that occurat particular points in time during the movie, including, withoutlimitation, color effects, 3D object effects, environment vibrationeffects, and screen size, location, and vibration effects.

At step 604, 3D objects/effects generator 332 executing on client device104 requests a particular 2D media content item. At step 606, contentmetadata collection engine 232 retrieves the virtual environment data.At step 608, content metadata collection engine 232 generates 3D contentmetadata based on the virtual environment data. At step 610, contentmetadata collection engine 232 transmits the 3D content metadata toclient device 104. At step 612, content metadata collection engine 232retrieves the 2D media content. At step 614, content metadata collectionengine 232 transmits the 2D media content to client device 104.

At step 616, 3D object/effects generator 332 retrieves 3D objects from3D object database as specified by the 3D content metadata. At step 618,3D objects/effects generator 332 generates the 3D virtual environment,virtual screen and movie buddy. The user could select the 3D virtualenvironment and movie buddy, such as a favorite character andenvironment, as default selections. In addition or alternatively, the 3Dcontent metadata may identify an environment and a movie buddy that isspecific to the 2D media content, such as an environment and a characterfrom the currently selected 2D media content. If neither the user northe 3D content metadata have specified an environment and a movie buddy,then 3D object/effects generator 332 selects a generic or defaultenvironment and movie buddy.

At step 620, 3D object/effects generator 332 displays the 2D mediacontent on the virtual screen. At step 622, 3D object/effects generator332 determines whether user input is detected. For example, 3Dobject/effects generator 332 could detect that the user has saidsomething to the movie buddy or has turned to look at the movie buddy.If 3D object/effects generator 332 has not detected user input, then themethod proceeds to step 626, described below. If, however, 3Dobject/effects generator 332 has detected user input, then the methodproceeds to step 624, where 3D object/effects generator 332 animates themovie buddy to respond to the user. For example, the movie buddy couldspeak to the user or nod his or her head at the user.

At step 626, 3D object/effects generator 332 determines whether aninactivity timer has expired. For example, 3D object/effects generator332 could determine, via an inactivity timer, that no user activity hasbeen detected for thirty minutes. If the inactivity timer has notexpired, then the method proceeds to step 630, described below. If,however, the inactivity timer has expired, then the method proceeds tostep 628, where 3D object/effects generator 332 animates the movie buddyto ask the user if he or she is still watching the 2D media content. 3Dobject/effects generator 332 could then continue playing the 2D mediacontent or terminate play as appropriate.

At step 630, 3D object/effects generator 332 determines whether a timedevent is detected. For example, 3D object/effects generator 332 coulddetect that the current time of the 2D media content is associated withone or more effects, such as a color, location, size, or vibrationeffect or an effect for animated objects traveling within the 3D virtualenvironment. If 3D object/effects generator 332 has not detected a timedevent, then the method proceeds to step 634, described below. If,however, 3D object/effects generator 332 has detected a timed event,then the method proceeds to step 632, where 3D object/effects generator332 alters the 3D virtual environment, virtual screen, and movie buddyaccording to the 3D content metadata. For example, 3D object/effectsgenerator 332 could change the color of the environment, the size andlocation of the virtual screen, and the vibration of the virtual screenand the environment. In addition or alternatively, 3D object/effectsgenerator 332 could generate and display 3D objects, such as flyingdebris, that travel through the 3D virtual environment.

At step 634, 3D object/effects generator 332 determines whether playbackof the 2D media content is complete. If playback of the 2D media contentis not complete, then the method proceeds to step 622, described above.If playback of the 2D media content is complete, then the methodproceeds to step 636, where 3D object/effects generator 332 terminatesplay of the 2D media content. 3D object/effects generator 332 may thentake other appropriate actions, including, without limitation, fadingthe 3D virtual environment to black, animating the movie buddy to wavegoodbye and walk away, and displaying a default visual effect. Themethod 600 then terminates.

In sum, techniques are disclosed for generating an immersive andinteractive 3D virtual environment that includes 2D media content, suchas movies, television programs, and other video content. When viewingsuch 2D media content through a VR headset, 3D elements are generatedand displayed as well, thereby presenting the 2D media content as if thecontent was originally created for a 3D VR environment.

Further, the techniques include capabilities to customize the 3D virtualenvironment. For example, the 3D virtual environment could include amovie buddy to “watch” the 2D movie content along with the user. Themovie buddy may be selected by the user. Alternatively, if the user doesnot choose a movie buddy, a content-specific movie buddy is recommendedbased on the particular 2D media content being viewed. The movie buddywill notify the user of any errors and otherwise “interact” with theuser in an effort to increase guest engagement. For example, the moviebuddy could ask the user if he or she is still watching the 2D mediacontent. In response, the user could nod or shake his or her head inresponse.

Further, the techniques include generating a more immersive environmentwhen the user is viewing 2D media content. Again, the environment may beselected by the user or automatically selected based on the 2D mediacontent. In one example, if the guest is watching a movie about underseacreatures, the environment could be changed to an underwater scene sothe guest is watching the content under the sea. In another example, ifthe user is viewing a science fiction movie, the environment could bechanged to simulate viewing the movie from within a spaceship. Inaddition, the environment dynamically changes as the scene of the 2Dmedia content changes. For example, if the 2D media content includes anexplosion on screen, the environment dynamically reacts accordingly. Forexample, the scene could get darker to amplify the visuals or flyingdebris could be displayed in the 3D virtual environment. In this manner,2D media content is augmented with dynamically generated 3D virtualelements.

At least one advantage of the disclosed techniques is that 2D mediacontent, when viewed via a VR headset, is more interactive andcustomizable relative to prior approaches. More particularly, bypresenting appropriate 3D elements in the VR environment and providingcustomized selections of different aspects of the VR environment, theuser experience is more immersive and unique relative to priorapproaches. As a result, the user has the experience of being “inside”the 2D media content rather than just viewing the content from an“outsider's” perspective. These advantages represent one or moretechnological improvements over the prior art.

1. In some embodiments, a computer-implemented method for generating athree-dimensional virtual environment includes receiving a media contentitem, receiving content metadata associated with the media content item,generating at least a portion of a 3D virtual environment based on thecontent metadata, and displaying the media content item and the at leasta portion of the 3D virtual environment on a display associated with avirtual reality device.

2. The computer-implemented method according to clause 1, wherein thecontent metadata includes an environment effect, and wherein generatingthe at least a portion of the 3D virtual environment based on thecontent metadata comprises receiving a first parameter that includes atleast one of an environment type and a uniform resource locater (URL)associated with the environment, and applying the environment effect tothe at least a portion of the 3D virtual environment based on the firstparameter.

3. The computer-implemented method according to clause 1 or clause 2,wherein the content metadata includes a movie buddy effect, and whereingenerating the at least a portion of the 3D virtual environment based onthe content metadata comprises receiving a first parameter that includesat least one of a movie buddy type and a uniform resource locater (URL)associated with the movie buddy, and applying the movie buddy effect tothe at least a portion of the 3D virtual environment based on the firstparameter.

4. The computer-implemented method according to any of clauses 1-3,wherein the content metadata includes a color effect, and whereingenerating the at least a portion of the 3D virtual environment based onthe content metadata comprises receiving a first parameter that includesat least one of a color value and a starting time, and applying thecolor effect to the at least a portion of the 3D virtual environmentbased on the first parameter.

5. The computer-implemented method according to any of clauses 1-4,wherein the content metadata includes a screen size effect, and whereingenerating the at least a portion of the 3D virtual environment based onthe content metadata comprises receiving a first parameter that includesat least one of a size type and a starting time, and applying the screenlocation effect to a virtual screen to include in the at least a portionof the 3D virtual environment based on the first parameter.

6. The computer-implemented method according to any of clauses 1-5,wherein the content metadata includes a three-dimensional (3D) objecteffect, and wherein generating the at least a portion of the 3D virtualenvironment based on the content metadata comprises receiving a firstparameter that includes at least one of an effect type, a starting time,a duration, and a uniform resource locater (URL) associated with the 3Dobject effect, and applying the 3D object effect to a 3D object toinclude in the at least a portion of the 3D virtual environment based onthe first parameter.

7. The computer-implemented method according to any of clauses 1-6,wherein the at least a portion of the 3D virtual environment includes avirtual screen, and wherein displaying the media content item includesoverlaying the media content item on the virtual screen.

8. The computer-implemented method according to any of clauses 1-7,wherein the at least a portion of the 3D virtual environment includes a3D object that is related to the media content item.

9. The computer-implemented method according to any of clauses 1-8,further comprising receiving a description of the at least a portion ofa 3D virtual environment in a first format, and translating thedescription of the at least a portion of a 3D virtual environment into asecond format that is native to the virtual reality device.

10. In some embodiments, a non-transitory computer-readable storagemedium includes instructions that, when executed by a processor, causethe processor to generate a three-dimensional virtual environment, byperforming the steps of receiving a media content item, receivingcontent metadata associated with the media content item, generating atleast a portion of a 3D virtual environment based on the contentmetadata, and displaying the media content item and the at least aportion of the 3D virtual environment on a display associated with avirtual reality device.

11. The non-transitory computer-readable storage medium according toclause 10, wherein the content metadata includes an environment effect,and wherein generating the at least a portion of the 3D virtualenvironment based on the content metadata comprises receiving a firstparameter that includes at least one of an environment type and auniform resource locater (URL) associated with the environment, andapplying the environment effect to the at least a portion of the 3Dvirtual environment based on the first parameter.

12. The non-transitory computer-readable storage medium according toclause 10 or clause 11, wherein the content metadata includes a moviebuddy effect, and wherein generating the at least a portion of the 3Dvirtual environment based on the content metadata comprises receiving afirst parameter that includes at least one of a movie buddy type and auniform resource locater (URL) associated with the movie buddy, andapplying the movie buddy effect to the at least a portion of the 3Dvirtual environment based on the first parameter.

13. The non-transitory computer-readable storage medium according to anyof clauses 10-12, wherein the content metadata includes a color effect,and wherein generating the at least a portion of the 3D virtualenvironment based on the content metadata comprises receiving a firstparameter that includes at least one of a color value and a startingtime, and applying the color effect to the at least a portion of the 3Dvirtual environment based on the first parameter.

14. The non-transitory computer-readable storage medium according to anyof clauses 10-13, wherein the content metadata includes an environmentvibration effect, and wherein generating the at least a portion of the3D virtual environment based on the content metadata comprises receivinga first parameter that includes at least one of a motion type and astarting time, and applying the environment vibration effect to the atleast a portion of the 3D virtual environment based on the firstparameter.

15. The non-transitory computer-readable storage medium according to anyof clauses 10-14, wherein the content metadata includes a screenvibration effect, and wherein generating the at least a portion of the3D virtual environment based on the content metadata comprises receivinga first parameter that includes at least one of a motion type and astarting time, and applying the screen vibration effect to a virtualscreen included in the at least a portion of the 3D virtual environmentbased on the first parameter.

16. The non-transitory computer-readable storage medium according to anyof clauses 10-15, wherein the content metadata includes a screenlocation effect, and wherein generating the at least a portion of the 3Dvirtual environment based on the content metadata comprises receiving afirst parameter that includes at least one of a location type and astarting time, and applying the screen location effect to a virtualscreen to include in the at least a portion of the 3D virtualenvironment based on the first parameter.

17. The non-transitory computer-readable storage medium according to anyof clauses 10-16, wherein the content metadata includes a screen sizeeffect, and wherein generating the at least a portion of the 3D virtualenvironment based on the content metadata comprises receiving a firstparameter that includes at least one of a size type and a starting time,and applying the screen location effect to a virtual screen to includein the at least a portion of the 3D virtual environment based on thefirst parameter.

18. The non-transitory computer-readable storage medium according to anyof clauses 10-17, wherein the content metadata includes athree-dimensional (3D) object effect, and wherein generating the atleast a portion of the 3D virtual environment based on the contentmetadata comprises receiving a first parameter that includes at leastone of an effect type, a starting time, a duration, and a uniformresource locater (URL) associated with the 3D object effect, andapplying the 3D object effect to a 3D object to include in the at leasta portion of the 3D virtual environment based on the first parameter.

19. In some embodiments, a computing device, comprises a memory thatincludes an three-dimensional (3D) object/effects generator, and aprocessor that is coupled to the memory and, when executing the 3Dobject/effects generator, is configured to: receive a media content itemfrom a content management server, receive, from the content managementserver, content metadata associated with the media content item, andgenerate at least a portion of a 3D virtual environment based on thecontent metadata.

20. The computing device according to clause 19, wherein the processor,when executing the 3D object/effects generator, is further configured toreceive a description of the at least a portion of a 3D virtualenvironment in a first format, and translate the description of the atleast a portion of a 3D virtual environment into a second format that isnative to the virtual reality device, and wherein the memory furtherincludes a native virtual reality engine, and the processor, whenexecuting the native virtual reality engine, is configured to display,via the second format, the media content item and the at least a portionof the 3D virtual environment on a display associated with a virtualreality device.

Any and all combinations of any of the claim elements recited in any ofthe claims and/or any elements described in this application, in anyfashion, fall within the contemplated scope of the present invention andprotection.

The descriptions of the various embodiments have been presented forpurposes of illustration, but are not intended to be exhaustive orlimited to the embodiments disclosed. Many modifications and variationswill be apparent to those of ordinary skill in the art without departingfrom the scope and spirit of the described embodiments.

Aspects of the present embodiments may be embodied as a system, methodor computer program product. Accordingly, aspects of the presentdisclosure may take the form of an entirely hardware embodiment, anentirely software embodiment (including firmware, resident software,micro-code, etc.) or an embodiment combining software and hardwareaspects that may all generally be referred to herein as a “module” or“system.” Furthermore, aspects of the present disclosure may take theform of a computer program product embodied in one or more computerreadable medium(s) having computer readable program code embodiedthereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

Aspects of the present disclosure are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thedisclosure. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, enable the implementation of the functions/acts specified inthe flowchart and/or block diagram block or blocks. Such processors maybe, without limitation, general purpose processors, special-purposeprocessors, application-specific processors, or field-programmable

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

While the preceding is directed to embodiments of the presentdisclosure, other and further embodiments of the disclosure may bedevised without departing from the basic scope thereof, and the scopethereof is determined by the claims that follow.

1. A computer-implemented method for generating a three-dimensional (3D)virtual environment, the method comprising: receiving a media contentitem; receiving content metadata associated with the media content item,wherein the content metadata specifies a movie buddy effect in which amovie buddy watches the media content item and interacts with a user inat least a portion of a 3D virtual environment; generating the at leasta portion of the 3D virtual environment based on the content metadata;and displaying the media content item and the at least a portion of the3D virtual environment on a display associated with a virtual realitydevice.
 2. The computer-implemented method of claim 1, wherein thecontent metadata further specifies an environment effect, and whereingenerating the at least a portion of the 3D virtual environment based onthe content metadata comprises: receiving a first parameter thatincludes at least one of an environment type or a uniform resourcelocater (URL) associated with the at least a portion of the 3D virtualenvironment; and applying the environment effect to the at least aportion of the 3D virtual environment based on the first parameter. 3.The computer-implemented method of claim 1, wherein generating the atleast a portion of the 3D virtual environment based on the contentmetadata comprises: receiving a first parameter that includes at leastone of a movie buddy type or a uniform resource locater (URL) associatedwith the movie buddy; and applying the movie buddy effect to the atleast a portion of the 3D virtual environment based on the firstparameter.
 4. The computer-implemented method of claim 1, wherein thecontent metadata further specifies a color effect, and whereingenerating the at least a portion of the 3D virtual environment based onthe content metadata comprises: receiving a first parameter thatincludes at least one of a color value or a starting time; and applyingthe color effect to the at least a portion of the 3D virtual environmentbased on the first parameter.
 5. The computer-implemented method ofclaim 1, wherein the content metadata further specifies a screen sizeeffect, and wherein generating the at least a portion of the 3D virtualenvironment based on the content metadata comprises: receiving a firstparameter that includes at least one of a size type or a starting time;and applying the screen size effect to a virtual screen to include inthe at least a portion of the 3D virtual environment based on the firstparameter.
 6. The computer-implemented method of claim 1, wherein thecontent metadata further specifies a 3D object effect, and whereingenerating the at least a portion of the 3D virtual environment based onthe content metadata comprises: receiving a first parameter thatincludes at least one of an effect type, a starting time, a duration, ora uniform resource locater (URL) associated with the 3D object effect;and applying the 3D object effect to a 3D object to include in the atleast a portion of the 3D virtual environment based on the firstparameter.
 7. The computer-implemented method of claim 1, wherein the atleast a portion of the 3D virtual environment includes a virtual screen,and wherein displaying the media content item includes overlaying themedia content item on the virtual screen.
 8. The computer-implementedmethod of claim 1, wherein the at least a portion of the 3D virtualenvironment includes a 3D object that is related to the media contentitem.
 9. The computer-implemented method of claim 1, further comprising:receiving a description of the at least a portion of the 3D virtualenvironment in a first format; and translating the description of the atleast a portion of the 3D virtual environment into a second format thatis native to the virtual reality device.
 10. A non-transitorycomputer-readable storage medium including instructions that, whenexecuted by a processor, cause the processor to generate athree-dimensional (3D) virtual environment, by performing the steps of:receiving a media content item; receiving content metadata associatedwith the media content item, wherein the content metadata specifies amovie buddy effect in which a movie buddy watches the media content itemand interacts with a user in at least a portion of a 3D virtualenvironment; generating the at least a portion of the 3D virtualenvironment based on the content metadata; and displaying the mediacontent item and the at least a portion of the 3D virtual environment ona display associated with a virtual reality device.
 11. Thenon-transitory computer-readable storage medium of claim 10, wherein thecontent metadata further specifies an environment effect, and whereingenerating the at least a portion of the 3D virtual environment based onthe content metadata comprises: receiving a first parameter thatincludes at least one of an environment type or a uniform resourcelocater (URL) associated with the at least a portion of the 3D virtualenvironment; and applying the environment effect to the at least aportion of the 3D virtual environment based on the first parameter. 12.The non-transitory computer-readable storage medium of claim 10, whereingenerating the at least a portion of the 3D virtual environment based onthe content metadata comprises: receiving a first parameter thatincludes at least one of a movie buddy type or a uniform resourcelocater (URL) associated with the movie buddy; and applying the moviebuddy effect to the at least a portion of the 3D virtual environmentbased on the first parameter.
 13. The non-transitory computer-readablestorage medium of claim 10, wherein the content metadata furtherspecifies a color effect, and wherein generating the at least a portionof the 3D virtual environment based on the content metadata comprises:receiving a first parameter that includes at least one of a color valueor a starting time; and applying the color effect to the at least aportion of the 3D virtual environment based on the first parameter. 14.The non-transitory computer-readable storage medium of claim 10, whereinthe content metadata further specifies an environment vibration effect,and wherein generating the at least a portion of the 3D virtualenvironment based on the content metadata comprises: receiving a firstparameter that includes at least one of a motion type or a startingtime; and applying the environment vibration effect to the at least aportion of the 3D virtual environment based on the first parameter. 15.The non-transitory computer-readable storage medium of claim 10, whereinthe content metadata further specifies a screen vibration effect, andwherein generating the at least a portion of the 3D virtual environmentbased on the content metadata comprises: receiving a first parameterthat includes at least one of a motion type or a starting time; andapplying the screen vibration effect to a virtual screen included in theat least a portion of the 3D virtual environment based on the firstparameter.
 16. The non-transitory computer-readable storage medium ofclaim 10, wherein the content metadata further specifies a screenlocation effect, and wherein generating the at least a portion of the 3Dvirtual environment based on the content metadata comprises: receiving afirst parameter that includes at least one of a location type or astarting time; and applying the screen location effect to a virtualscreen to include in the at least a portion of the 3D virtualenvironment based on the first parameter.
 17. The non-transitorycomputer-readable storage medium of claim 10, wherein the contentmetadata further specifies a screen size effect, and wherein generatingthe at least a portion of the 3D virtual environment based on thecontent metadata comprises: receiving a first parameter that includes atleast one of a size type r a starting time; and applying the screen sizeeffect to a virtual screen to include in the at least a portion of the3D virtual environment based on the first parameter.
 18. Thenon-transitory computer-readable storage medium of claim 10, wherein thecontent metadata further specifies a 3D object effect, and whereingenerating the at least a portion of the 3D virtual environment based onthe content metadata comprises: receiving a first parameter thatincludes at least one of an effect type, a starting time, a duration, ora uniform resource locater (URL) associated with the 3D object effect;and applying the 3D object effect to a 3D object to include in the atleast a portion of the 3D virtual environment based on the firstparameter.
 19. A computing device, comprising: a memory that includes athree-dimensional (3D) object/effects generator; and a processor that iscoupled to the memory and, when executing the 3D object/effectsgenerator, is configured to: receive a media content item from a contentmanagement server; receive, from the content management server, contentmetadata associated with the media content item, wherein the contentmetadata specifies a movie buddy effect in which a movie buddy watchesthe media content item and interacts with a user in at least a portionof a 3D virtual environment; and generate the at least a portion of the3D virtual environment based on the content metadata.
 20. The computingdevice of claim 19, wherein the processor, when executing the 3Dobject/effects generator, is further configured to: receive adescription of the at least a portion of the 3D virtual environment in afirst format; and translate the description of the at least a portion ofthe 3D virtual environment into a second format that is native to avirtual reality device, wherein the memory further includes a nativevirtual reality engine, and the processor, when executing the nativevirtual reality engine, is configured to display, via the second format,the media content item and the at least a portion of the 3D virtualenvironment on a display associated with the virtual reality device.